GAO 


United  States  Government  Accountability  Office _ 

Report  to  Congressional  Committees 


November  2015 


DEFENSE 
ADVANCED 
RESEARCH 
PROJECTS  AGENCY 


Key  Factors  Drive 
Transition  of 
Technologies,  but 
Better  Training  and 
Data  Dissemination 
Can  Increase 
Success 


GAO-16-5 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

NOV  2015  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2015  to  00-00-2015 

4.  TITLE  AND  SUBTITLE 

Defense  Advanced  Research  Projects  Agency:  Key  Factors  Drive 
Transition  of  Technologies,  but  Better  Training  and  Data  Dissemination 
Can  Increase  Success 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Government  Accountability  Office, 441  G  Street 

NW, Washington, DC, 20548 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

45 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


November  2015 


GAD 

Highlights 

Highlights  of  GAO-16-5,  a  report  to 
congressional  committees 


DEFENSE  ADVANCED  RESEARCH  PROJECTS 
AGENCY 

Key  Factors  Drive  Transition  of  Technologies,  but 
Better  Training  and  Data  Dissemination  Can  Increase 
Success 


Why  GAO  Did  This  Study 

After  the  Soviet  Union  launched  the 
first  satellite  into  orbit  in  1957,  the  U.S. 
government  made  a  commitment  to 
initiate,  rather  than  react  to,  strategic 
technological  surprises.  DOD  relies  on 
DARPA’s  disruptive  innovations  to 
maintain  this  promise,  backed  by 
congressional  appropriations  of  over 
$2.9  billion  in  fiscal  year  2015  alone.  In 
April  2015,  DOD  reported  that  U.S. 
technological  superiority  is  again  being 
challenged  by  potential  adversaries 
and  renewed  efforts  to  improve  its 
products.  Meanwhile,  GAO  found 
deficiencies  in  DOD’s  technology 
transition  processes  that  may  hinder 
these  efforts  and  DARPA’s  goals. 

Senate  Report  113-176  included  a 
provision  for  GAO  to  review  DOD’s 
technology  transition  processes, 
practices,  and  results.  This  report 
focuses  on  DARPA  and  assesses  its 
(1)  effectiveness  at  transitioning 
technologies  since  fiscal  year  2010, 
including  identifying  factors  that 
contribute  to  successful  transitions, 
and  (2)  implementation  of  DOD 
policies  and  programs  intended  to 
facilitate  technology  transition.  GAO 
reviewed  DARPA  programs  completed 
since  2010;  identified  transition  factors 
by  analyzing  program  documentation 
for  a  random  sample  of  10  cases; 
reviewed  DOD  policies;  and 
interviewed  DOD  officials. 

What  GAO  Recommends 

DARPA  should  regularly  assess 
technology  transition  strategies,  refine 
training  requirements,  and  increase 
dissemination  of  technical  data  for 
completed  programs.  DOD  did  not 
agree  to  take  GAO’s  recommended 
actions,  which  remain  warranted,  as 
discussed  in  the  report. 

View  GAO-16-5.  For  more  information,  contact 
Michael  J.  Sullivan  at  (202)  512-4841  or 
sullivanm@gao.gov. 


What  GAO  Found 


Since  2010,  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  has  had 
success  in  technology  transition — the  process  of  migrating  new  technologies 
from  the  research  environment  to  military  users,  including  Department  of 
Defense  (DOD)  acquisition  programs  and  warfighters.  However,  inconsistencies 
in  how  the  agency  defines  and  assesses  its  transition  outcomes  preclude  GAO 
from  reliably  reporting  on  transition  performance  across  DARPA’s  portfolio  of  150 
programs  that  were  successfully  completed  between  fiscal  years  2010  and  2014. 
These  inconsistencies  are  due  in  part  to  shortfalls  in  agency  processes  for 
tracking  technology  transition.  Nevertheless,  GAO’s  analysis  of  10  selected 
programs  identified  four  factors  that  contributed  to  transition  success,  the  most 
important  being  military  or  commercial  demand  for  the  planned  technology  and 
linkage  to  a  research  area  where  DARPA  has  sustained  interest.  Both  of  these 
factors  were  generally  evident  at  the  time  a  program  started,  while  the  other  two 
factors  were  observed  later,  once  the  program  was  underway.  The  figure  below 
highlights  the  four  factors. 
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Source:  GAO  analysis  of  Defense  Advanced  Research  Projects  Agency  (DARPA)  information  |  GAO-16-5 


DARPA's  implementation  of  DOD  programs  intended  to  foster  technology 
transition  has  been  limited  and  neither  DOD  nor  DARPA  have  defined  policies  for 
managing  transition  activities.  DARPA  has  also  largely  elected  not  to  participate 
in  DOD  technology  transition  programs,  with  the  exception  of  federally  mandated 
small  business  programs,  citing  challenges  in  meeting  program  requirements 
within  DARPA's  typical  three-  to  five-year  timeframe  for  executing  its  research 
initiatives.  Instead,  DARPA  primarily  focuses  its  time  and  resources  on  creating 
radically  innovative  technologies  that  support  DOD's  warfighting  mission  and 
relegates  technology  transition  to  a  secondary  priority.  DARPA  leadership  defers 
to  its  program  managers  to  foster  technology  transition,  but  provides  limited 
related  training.  Moreover,  while  its  leadership  conducts  oversight  of  program 
managers’  activities  through  periodic  program  reviews,  these  reviews  do  not 
regularly  assess  technology  transition  strategies.  GAO  has  found  that  this 
approach  does  not  consistently  position  programs  for  transition  success.  Further, 
while  DARPA  disseminates  information  on  its  past  programs  within  DOD,  to  the 
public,  and  among  private  companies,  it  does  not  take  full  advantage  of 
government-sponsored  resources  for  sharing  technical  data,  which  may  obscure 
visibility  into  its  programs  and  lead  to  missed  transition  opportunities. 
_ United  States  Government  Accountability  Office 
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GAO 

441  G  St.  N.W. 
Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


November  18,  2015 
Congressional  Committees 

The  Defense  Advanced  Research  Projects  Agency  (DARPA)  is  a 
Department  of  Defense  (DOD)  agency  whose  mission  is  to  create  new, 
disruptive  technologies  in  the  interest  of  national  security.  Its  genesis 
dates  to  the  Soviet  Union’s  launch  of  the  first  satellite,  Sputnik  1,  into  orbit 
in  1957,  which  spurred  a  commitment  by  the  U.S.  government  that,  from 
that  time  forward,  it  would  initiate,  rather  than  react  to,  strategic 
technological  surprises.  DOD  relies  on  technological  innovation  produced 
by  DARPA  to  maintain  this  promise — and  ensure  superiority  of  the 
nation’s  weapon  systems  and  armed  forces — to  protect  U.S.  interests 
both  at  home  and  abroad.  In  fiscal  year  2015,  Congress  appropriated 
over  $2.9  billion  to  DARPA  to  support  its  mission  and  objectives,  which 
include  basic  and  applied  research  activities,  as  well  as  advanced 
technology  development. 

In  April  2015,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics  reported  concern  that  U.S.  technological  superiority  is  being 
challenged  by  potential  adversaries  in  ways  not  seen  since  the  Cold 
War.1  As  a  result,  the  Under  Secretary  is  leading  new  efforts  aimed  at 
strengthening  DOD  innovation  and  technical  excellence  and  the  ability  of 
DOD  products  to  continue  providing  military  technological  superiority. 
However,  we  have  previously  found  deficiencies  in  DOD’s  processes  for 
migrating  new  technologies  from  the  laboratory  or  research  environment 
to  the  acquisition  and  warfighter  communities — a  pursuit  known  as 
technology  transition — that  may  undermine  the  Under  Secretary’s  efforts 


1 0ffice  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
Memorandum,  Subject:  Implementation  Directive  for  Better  Buying  Power  3.0 — Achieving 
Dominant  Capabilities  through  Technical  Excellence  and  Innovation  (Washington,  D.C.: 
Apr.  9,  2015). 
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and  DARPA’s  goals.2  Congress  also  recently  expressed  concern 
regarding  barriers  to  technology  transition.  Senate  Report  113-176  to  the 
National  Defense  Authorization  Act  for  Fiscal  Year  2015  included  a 
provision  that  GAO  review  DARPA  and  other  DOD  technology  transition 
processes,  practices,  and  results.3  Our  work  focused  exclusively  on 
DARPA  and  assessed  the  agency’s  (1)  effectiveness  at  transitioning 
technologies  since  fiscal  year  2010,  including  identifying  the  factors  that 
contributed  to  successful  technology  transitions,  and  (2)  implementation 
of  DOD  policies  and  programs  intended  to  facilitate  the  transition  of 
technologies. 

To  assess  DARPA’s  effectiveness  at  transitioning  technologies,  including 
identifying  the  factors  that  contribute  to  transitions,  we  reviewed  data  on 
150  programs  that  DARPA  identified  as  having  completed  as  planned  and 
producing  a  substantive  technological  gain  or  innovation  for  fiscal  years 
2010  through  2014,  regardless  of  whether  that  technology  or  innovation 
transitioned  to  an  end  user.4  We  confined  our  analysis  to  this  time  frame 
owing  to  availability  of  data  from  DARPA.  Using  this  portfolio-level  data 
set,  we  randomly  selected  10  programs — 5  that  transitioned  and  5  that 
did  not — for  case  study  analyses.  We  then  analyzed  relevant  program- 
level  documentation  for  our  selected  cases  to  identify  factors  that  facilitate 
transition  success.  While  reviewing  these  case  study  programs,  we 
identified  inconsistencies  between  agency  portfolio-level  transition 
outcome  data  and  program-level  information.  As  a  result,  we  concluded 
that  DARPA’s  portfolio-level  data  were  not  sufficiently  reliable  for  the 
purposes  of  assessing  agency-wide  transition  rates  and  outcomes  since 
fiscal  year  2010.  However,  these  inconsistencies  did  not  significantly 
affect  our  program  selections;  therefore,  these  data  were  sufficiently 


2GAO,  Defense  Technology  Development:  Management  Processes  Can  Be  Strengthened 
for  New  Technology  Transition  Programs,  GAO-05-480  (Washington,  D.C.:  June  17, 
2005);  Best  Practices:  Stronger  Practices  Needed  to  Improve  DOD  Technology  Transition 
Processes,  GAO-06-883  (Washington,  D.C.:  Sept.  14,  2006);  and  Defense  Technology 
Development:  Technology  Transition  Programs  Support  Military  Users,  but  Opportunities 
Exist  to  Improve  Measurement  of  Outcomes,  GAO-13-286  (Washington,  D.C.:  Mar.  7, 
2013). 

3S.  Rep.  No.  113-176,  at  60-61  (2014). 

4The  data  covered  programs  funded  under  DOD  budget  activities  6.2  (applied  research) 
and  6.3  (advanced  technology  development)  that  DARPA  identified  as  having  completed 
as  planned  and  producing  a  substantive  technological  gain  or  innovation.  We  interviewed 
knowledgeable  officials  and  reviewed  existing  documentation  about  the  data. 
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reliable  for  case  study  selection  purposes.  We  then  analyzed  DARPA 
provided  documentation — including  program  briefings,  memorandums  of 
agreement,  and  program  completion  reports — for  our  selected  programs 
to  identify  factors  that  facilitated  or  precluded  their  individual  transitions. 
We  then  conducted  a  content  analysis  of  these  individual  factors  to 
identify  common  themes  among  the  programs,  which  led  us  to  identifying 
the  four  significant  factors  that  underpinned  transition  outcomes  in  the 
programs  we  reviewed. 

To  assess  DARPA’s  implementation  of  DOD  policies  and  programs 
intended  to  facilitate  the  transition  of  technologies,  we  analyzed  policy 
instructions,  guidance,  training  materials,  and  technical  data  repositories 
intended  to  promote  technology  transition.  We  also  reviewed  our  prior 
reports  and  DOD  documentation  on  the  Small  Business  Innovation 
Research  (SBIR)  and  Small  Business  Technology  Transfer  (STTR) 
programs  and  other  DOD-wide  initiatives  aimed  at  facilitating  technology 
transition  to  understand  the  extent  to  which  DARPA  participates  in  these 
programs.  We  supplemented  the  above  analyses  through  interviews  with 
current  and  former  officials  from  DARPA,  the  Office  of  the  Secretary  of 
Defense,  selected  military  service  acquisition  and  requirements  offices, 
and  selected  DOD  research  centers. 

We  conducted  this  performance  audit  from  January  2015  to  November 
2015  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


Background 


DOD  Technology  DOD  invests  about  $12  billion  in  funding  to  support  its  science  and 

Management  Process  technology  community,  which  it  relies  upon  to  identify,  pursue,  and 

develop  new  technologies  to  improve  and  enhance  military  capabilities. 
This  community  is  comprised  of  DOD-wide  research  agencies,  including 
DARPA,  as  well  as  military  service  research  agencies  and  laboratories, 
test  facilities,  private  industry,  and  academic  institutions,  and  is  overseen 
by  the  Office  of  the  Assistant  Secretary  of  Defense  for  Research  and 
Engineering.  The  research  and  development  activities  these  different 
components  engage  in  are  intended  to  produce  mature  technologies  that 
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DOD  can  integrate  and  deliver  in  systems  that  support  its  warfighters. 

This  integration  process,  known  as  product  development,  represents  the 
handover  of  breakthrough  technologies  from  DOD’s  science  and 
technology  community  to  its  acquisition  community.  Although  not 
precisely  defined,  technology  transition  generally  occurs  at  the  point  when 
advanced  technology  development  ends  and  this  new  product 
development  begins.  Figure  1  illustrates  DOD’s  technology  management 
process. 


Figure  1:  DOD  Technology  Management  Process 


Technology  Investment 


Technology  Development 


Product  Development 


Technology  shelved 
but  available  for  use 


Technology 
integrated  into 
acquisition 
program  or  fielded 


Source:  GAO  analysis  of  Department  of  Defense  science  &  technology  management  process.  |  GAO-16-5 


DOD  has  long  noted  the  existence  of  a  chasm  between  its  science  and 
technology  community  and  its  acquisition  community  that  impedes 
technology  transition  from  consistently  occurring.  This  chasm,  often 
referred  to  by  department  insiders  as  “the  valley  of  death,”  exists  because 
the  acquisition  community  often  requires  a  higher  level  of  technology 
maturity  than  the  science  and  technology  community  is  willing  to  fund  and 
develop.  In  2007,  DOD  reported  that  this  gap  can  only  be  bridged  through 
cooperative  efforts  and  investments  from  both  communities,  such  as  early 
and  frequent  collaboration  among  the  developer,  acquirer,  and  user.5 


5DOD,  Department  of  Defense  Report  to  Congress  on  Technology  Transition 
(Washington,  D.C.:  July  2007). 


Page  4 


GAO-16-5  DARPA  Technology  Transition 


We  have  also  reported  extensively  on  shortfalls  across  DOD’s  technology 
management  enterprise  in  transitioning  technologies  from  development  to 
acquisition  and  fielding.  In  June  2005,  we  found  that  DOD  technology 
transition  programs  faced  challenges  selecting,  managing,  and 
overseeing  projects,  and  assessing  outcomes.6  In  September  2006,  we 
found  that  DOD  lacked  the  key  planning,  processes,  and  metrics  used  by 
leading  commercial  companies  to  successfully  develop  and  transition 
technologies.7  More  recently,  in  March  2013,  we  found  that  the  vast 
majority  of  DOD  technology  transition  programs  provide  technologies  to 
military  users,  but  tracking  of  project  outcomes  and  other  benefits  derived 
after  transition  remained  limited.8 


DARPA  Processes  and 
Programs 


DARPA’s  scientific  investigations  run  the  gamut  from  laboratory  efforts  to 
the  creation  of  full-scale  technology  demonstrations  in  the  fields  of 
biology,  medicine,  computer  science,  chemistry,  physics,  engineering, 
mathematics,  material  sciences,  social  sciences,  neurosciences,  and 
more.  The  agency  solicits  proposals  for  research  work  in  support  of  its 
scientific  endeavors  through  broad  agency  announcements.9  These 
solicitations  seek  thought  leaders  and  technological  pioneers  that  can 
leverage  new  ideas  in  science  to  advance  the  state  of  the  art  beyond  the 
practical  application  of  knowledge.  Non-DARPA  entities  respond  to  broad 
agency  announcements  by  submitting  proposals  for  executing  work  to 
meet  the  agency’s  stated  needs.  DARPA  reviews  those  proposals  based 
on  technical  merit,  and  entities  receiving  awards  are  thereafter  referred  to 
as  performers. 

To  execute  solicitations,  awards,  and  program  oversight,  DARPA  relies 
on  approximately  220  government  employees,  including  nearly  100 
program  managers.  Program  managers  report  to  DARPA’s  office 
directors  and  their  deputies,  who  are  responsible  for  charting  the  strategic 
directions  of  six  technical  offices.  The  technical  staff  is  supported  by 


6GAO-05-480. 

7GAO-06-883. 

8GAO-1 3-286. 

9A  broad  agency  announcement  is  a  competitive  solicitation  procedure,  which  may  be 
used  to  obtain  proposals  for  basic  and  applied  research  and  that  part  of  development  not 
related  to  the  development  of  a  specific  system  or  hardware  procurement.  See  Federal 
Acquisition  Regulation  §§  6.102(d)(2)  and  35.016. 
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experts  in  security,  legal  and  contracting  issues,  finance,  human 
resources,  and  communications.  DARPA’s  Director  and  Deputy  Director 
approve  new  programs  and  lead  scientific  and  technical  reviews  of 
ongoing  programs,  while  setting  agency-wide  priorities  and  ensuring  a 
balanced  investment  portfolio.  Currently,  DARPA  has  about  250  ongoing 
research  and  development  programs  in  its  portfolio. 

The  10  recently  completed  programs  that  we  reviewed  for  this  report 
together  spanned  a  broad  range  of  research  areas,  including 
communications,  navigation,  and  health  and  marine  sciences.  Table  1 
highlights  the  research  focuses  of  these  10  programs  in  more  detail. 


Table  1:  Overview  of  DARPA  Programs  Selected  for  GAO  Case  Study  Analyses 

Program  name 

Program  description 

Advanced  Wireless  Networks  for  Soldier 

The  program  sought  to  develop  a  cost-efficient  military  radio  system  that  integrated 
commercial  hardware  with  the  Army’s  radio  waveform  and  delivered  improved  capability, 
including  the  ability  to  dynamically  identify  available  radio  spectrum  and  transmit  on  it. 

Architecture  for  Diode  High  Energy  Laser 
Systems 

The  program  developed  technologies  for  combining  different  laser  beams  together  in 
order  to  generate  laser  output  powers  needed  to  support  military  applications,  but  with 
ultra  low  size,  weight,  and  power  attributes. 

Dynamic  Prevention  of  Biofouling 

The  program  sought  to  develop  surfaces  and  coatings  that  resist  biofouling  for  extended 
periods — without  use  of  chemical  substances  or  microorganisms — in  the  static  marine 
environments  in  which  Navy  ships  often  operate. 

Falcon  Combined-cycle  Engine 

Technology 

The  program  developed  advanced  hypersonic  turbine  engine  technologies  for  use  with  a 
separately  developed  hypersonic  cruise  vehicle. 

Nastic  Materials 

The  program  developed  controllable,  active  materials  capable  of  changing  shapes  to 
adapt  to  different  environments,  similar  to  the  way  plants  move  under  different  strains 
and  forces. 

Predicting  Health  and  Disease 

The  program  developed  a  predictive  model  and  diagnostic  test  platform  for  detecting  pre- 
symptomatic  exposure  to  infectious  diseases  and  predicting  future  illness,  creating  the 
potential  for  early  preventative  treatment  and  control  of  viral  infections. 

Quint  Networking  Technology 

The  program  developed  and  demonstrated  modular  multi-band  network  data  link 
technologies  that  worked  across  five  nodes — aircraft,  unmanned  combat  air  vehicles, 
weapons,  tactical  unmanned  air  vehicles,  and  dismounted  ground  forces. 

Self-Regenerative  Systems 

The  program  sought  to  develop  technology  that  would  permit  military  computing  systems 
to  provide  critical  functionality  at  all  times,  despite  damage  caused  by  unintentional 
software  errors  or  malicious  attacks. 

Spoken  Language  Communication  and 
Translation  System  for  Tactical  Use 

The  program  developed  and  demonstrated  two-way  translation  systems  for  Iraqi  Arabic, 
Dari,  and  Pashto  languages  using  digital  platforms  of  various  sizes  and  forms. 

Tactical  Underwater  Navigation  System 

The  program  integrated  various  commercial  navigation  technologies  into  a  single  unit  that 
provided  divers  and  smaller  diving  propulsion  devices  with  an  accurate  and  economical 
system  for  navigating  underwater. 

Source:  GAO  analysis  of  DARPA  information.  |  GAO-16-5 
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Overall  Transition 
Rates  Not 
Determinable,  but 
Selected  Programs 
Illustrate  Factors  for 
Transition  Success 


Since  2010,  DARPA  has  had  success  in  transitioning  new  technologies 
from  the  research  environment  to  military  users,  including  DOD 
acquisition  programs  and  warfighters.  DARPA  maintains  a  portfolio-level 
database  that  identifies  these  outcomes  by  program.  However,  the 
agency’s  process  for  tracking  technology  transition  outcomes  is  not 
designed  to  capture  transitions  that  occur  after  a  program  completes  and 
does  not  provide  DARPA  with  an  effective  means  for  updating  its 
database.  We  used  outputs  from  this  database  to  select  10  case  study 
programs,  but  later  identified  inconsistencies  affecting  three  programs  in 
how  transition  outcomes  were  reported  in  the  portfolio-level  database 
versus  how  they  were  reported  in  other  program  documentation.  We  then 
concluded  that  DARPA’s  portfolio-level  database  was  unreliable  for 
assessing  transition  rates  and  outcomes  since  fiscal  year  2010.  Our 
analysis  of  the  10  selected  programs  did,  however,  identify  four  factors 
that  contributed  to  transition  successes,  the  most  important  of  which  were 
military  or  commercial  demand  for  the  planned  technology  and  linkage  to 
a  research  area  where  DARPA  has  sustained  interest. 


DARPA’s  Approach  to 
Developing  Technology 
Contributes  to  a  Broad 
Definition  of  Transition 
Success 


DARPA’s  technological  approach  focuses  on  radical  innovation  that 
addresses  future  warfighting  needs,  rather  than  developing  technologies 
that  address  current  warfighting  needs.  This  approach  shapes  how  the 
agency  defines,  pursues,  and  tracks  technology  transition.  DARPA 
considers  a  successful  transition  to  be  one  where  its  program,  or  a 
portion  of  its  program,  influences  or  introduces  new  knowledge.  This 
knowledge  is  often  passed  through  program  performers,  which  DARPA 
relies  on  to  execute  technology  development  in  its  programs.  Typical 
performers  include  commercial  enterprises;  other  DOD  entities,  such  as 
military  service  laboratories  and  research  agencies;  and  academic 
institutions.  Further,  DARPA  generally  does  not  develop  technologies  to 
full  maturity.  Instead,  the  agency  focuses  on  demonstrating  the  feasibility 
of  new  technologies,  which  includes  verifying  that  the  concepts  behind 
the  technologies  have  potential  for  real  life  applications.  As  a  result,  most 
DARPA  technologies  require  additional  development  before  they  are 
ready  for  operational  or  commercial  use.  Therefore,  follow-on 
development  is  the  predominant  path  of  technology  transition  at  DARPA. 
Table  2  highlights  the  different  technology  transition  paths  that  DARPA 
technologies  can  take. 
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Table  2:  Technology  Transition  Paths  Tracked  by  DARPA 

Transition  Path 

Description 

Enable  other  government  agency 

Another  federal  government  agency  outside  of  the  DOD  funds  and  manages  additional 
development  of  the  technology,  or  appropriates  the  DARPA  technology. 

Direct  to  operational  use 

DARPA  research  or  technology  transfers  directly  to  an  end  user  organization — such  as  a 
military  service,  another  DOD  agency,  or  other  federal,  state  or  local  organizations — and 
is  used  in  current  operations/missions. 

Program  of  record3 

The  DARPA  technology  is  transferred  to  another  DOD  component  for  further  development 
in  a  program  of  record. 

Commercialization 

DARPA  performers  or  other  commercial  entities  sell  the  developed  technology  to  the 
federal  government  or  on  the  commercial  market. 

DARPA  program  insertion 

Occurs  in  one  of  two  ways:  (1 )  performers  who  successfully  complete  technology 
development  in  one  DARPA  program  then  propose  work  on  another  DARPA  program 
using  solutions  demonstrated  in  the  earlier  program  or  (2)  DARPA  initiates  a  follow-on 
program  based  upon  or  including  the  technology  from  an  earlier  completed  program. 

Follow-on  development 

Performers  or  others  use  non-DOD  resources  to  further  develop  and  eventually  use, 
implement,  or  commercialize  the  technology  following  DARPA  program  completion. 

Follow-on  development  by  a  DOD 
component 

Another  DOD  component  funds  continued  development,  use,  or  implementation  of  the 
technology  following  completion  of  the  DARPA  program. 

Influences  or  establishes  a  defined 
technology  standard 

A  DARPA  funded  program  directly  leads  to  the  development  of  a  standard  or  defined 
technological  benchmark  within  the  science  and  technology  community. 

Source:  GAO  analysis  of  DARPA  information.  |  GAO-16-5 

aDOD  defines  a  program  of  record  as  an  acquisition  program  that  is  currently  funded  or  has 
successfully  achieved  formal  program  initiation. 

DARPA’s  definition  of  what  constitutes  technology  transition  reflects  one 
of  many  in  use  within  DOD.  In  June  2005,  the  Office  of  the  Deputy  Under 
Secretary  of  Defense  for  Advanced  Systems  and  Concepts10  in 
collaboration  with  the  Defense  Acquisition  University  (DAU)  published 
guidance  defining  technology  transition  as  “the  use  of  technology  in 
military  systems  to  create  effective  weapons  and  support  systems — in  the 
quantity  and  quality  needed  by  the  warfighter  to  carry  out  assigned 
missions  at  the  ‘best  value’  as  measured  by  the  warfighter.”11  However, 
DOD  officials  told  us  the  2005  guidance  is  outdated,  does  not  constitute 


1 0 

According  to  officials  from  the  Office  of  the  Secretary  of  Defense,  the  Office  of  the 
Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  and  Concepts  was 
disestablished  and  its  activities  have  since  transferred  to  the  Office  of  the  Deputy 
Assistant  Secretary  of  Defense  for  Emerging  Capability  and  Prototyping. 

11  DAU  and  the  Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems 
and  Concepts,  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment,  Version  2.0  (DOD/DAU  University  Press,  Fort  Belvoir,  Va.:  June  2005). 
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department  policy,  and  should  only  be  considered  as  a  useful  reference 
source.  In  the  absence  of  current  DOD  policy,  in  a  March  2013  report  we 
identified  three  communities  that  DOD  technologies  typically  transitioned 
to:  acquisition  programs;  directly  to  the  field  for  use  by  the  warfighter;  and 
to  other  users  such  as  science  and  technology  organizations,  test  and 
evaluation  centers,  or  industry.12  The  communities  we  identified  in  2013 
are  similar  to  the  transition  outcomes  listed  in  the  2005  guidance,  which 
broadly  lists  commercialization,  acquisition  program,  and  follow-on 
development  by  the  prime  contractor  as  primary  pathways  of  technology 
transition.  In  a  subsequent  report  in  December  2013,  we  found  further 
differences  among  what  the  military  services  define  as  technology 
transition  and  additional  confirmation  that  DOD  itself  lacks  a  formal 
definition  for  technology  transition  across  the  department.13  These 
variations,  in  tandem  with  the  absence  of  a  standard  DOD-wide  definition 
of  technology  transition,  prevents  the  military  services,  DOD  research 
agencies,  and  other  DOD  entities  from  consistently  defining  and  tracking 
technology  transition.  This  lack  of  a  formal  definition  of  technology 
transition  means  that  DOD  entities,  such  as  DARPA,  are  free  to  define 
and  categorize  technology  transition  for  themselves. 


DARPA’s  Tracking  of 
Technology  Transition 
Concludes  at  Program 
Completion,  Which  Limits 
Data  Reliability  and 
Precludes  Evaluation  of 
Transition  Rates 


Following  a  program’s  completion,  DARPA  officials  identify  and  record 
transition  outcomes  in  accordance  with  the  technology  transition  paths 
identified  in  table  2.  DARPA  collects  this  information  within  a  portfolio- 
level  database  that  spans  all  of  its  recently  completed  programs.  The 
agency  uses  this  database  primarily  to  provide  incoming  program 
managers  with  training  on  potential  transition  opportunities.  Figure  2 
illustrates  in  more  detail  DARPA’s  process  for  assessing  technology 
transition  outcomes  in  its  programs. 


12GAO-1 3-286. 

13GAO,  Small  Business  Innovation  Research:  DOD’s  Program  Supports  Weapon 
Systems,  but  Lacks  Comprehensive  Data  on  Technology  Transition  Outcomes, 
GAO-14-96  (Washington,  D.C.:  Dec.  20,  2013). 
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Figure  2:  DARPA  Process  for  Assessing  Technology  Transition  Outcomes 
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Source:  GAO  analysis  of  Defense  Advanced  Research  Projects  Agency  (DARPA)  information.  |  GAO-16-5 


DARPA’s  process  for  tracking  technology  transition  outcomes  is  not 
designed  to  capture  transitions  that  occur  after  a  program  completes  and 
the  agency’s  agreements  with  program  performers  have  ended.  After  this 
point,  however,  program  performers  often  continue  to  develop  their 
technologies  using  non-DARPA  sources  of  funding.  According  to  DARPA 
officials,  these  efforts  can  result  in  later  transitions  of  technologies  to 
commercial  products — including  ones  that  are  sold  back  to  DOD  for 
military  use — without  the  agency’s  knowledge.14 

This  process  for  tracking  technology  transition  outcomes  also  does  not 
provide  DARPA  with  an  effective  means  for  updating  its  portfolio-level 
database.  We  used  outputs  from  this  database  to  select  10  case  study 
programs  (5  that  transitioned  and  5  that  did  not  transition),  but  later 
identified  inconsistencies  affecting  three  programs  in  how  transition 
outcomes  were  reported  in  the  portfolio-level  database  versus  how  they 
were  reported  in  other  program  documentation  that  we  reviewed.  This 
confusion  about  ultimate  transition  outcomes  persisted  during  our 
interviews  with  DARPA  officials.  As  a  result,  we  concluded  that  DARPA’s 
portfolio-level  database  was  unreliable  for  assessing  transition  rates  and 
outcomes  since  fiscal  year  2010. 15  Table  3  highlights  the  inconsistencies 
we  found  in  our  reviews. 


14This  outcome  is  consistent  with  DARPA’s  approach  to  intellectual  property  rights  for 
non-sensitive  technologies,  which  allows  DOD  to  retain  government  purpose  rights  to  the 
technologies  and  program  performers  to  introduce  and  sell  the  innovation  within  the 
commercial  marketplace. 

15These  inconsistencies  did  not  significantly  affect  our  program  selections;  therefore,  the 
data  were  sufficiently  reliable  for  the  purposes  of  selecting  the  10  case  study  programs. 
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Table  3:  Technology  Transition  Data  Inconsistencies  among  Selected  DARPA  Programs 

Program  name 

According  to  DARPA  portfolio-level  database 

According  to  GAO  analysis 

Advanced  Wireless  Networks 
for  the  Soldier 

Non-transition:  program  completed  as  planned,  but  lacked 
sufficient  transition  interest  outside  DARPA 

Transition:  follow-on  development  by  a 
Department  of  Defense  (DOD) 
component 

Dynamic  Prevention  of 
Biofouling 

Transition:  follow-on  development  by  a  DOD  component 

Non-transition:  program  funding  was 
terminated  shortly  after  program 
initiation 

Self-Regenerative  Systems 

Non-transition:  program  achieved  its  technical  objectives, 
but  transition  partner(s)  desired  a  more  mature  technology 

Non-transition:  program  did  not 
achieve  its  technical  objectives 

Source:  GAO  analysis  of  DARPA  and  DOD  data  and  information.  |  GAO-16-5 


The  inconsistencies  we  identified  suggest  that  DARPA’s  current  approach 
to  tracking  technology  transitions  can  limit  its  understanding  of  transition 
outcomes.  This  may  undermine  its  ability  to  craft  transition  plans  for  new 
programs  based  on  the  lessons  learned  from  previous  programs.  We 
have  previously  identified  technology  transition  tracking  as  a  longstanding 
issue  at  DOD.  For  example,  in  September  2006,  we  found  that  tracking 
technology  transitions  and  the  effect  of  transitions,  such  as  cost  savings 
or  deployment  of  the  technology  in  a  product,  provided  key  feedback  that 
can  inform  the  future  management  of  programs.  However,  in  March  2013, 
we  found  that  DOD  stopped  tracking  transition  outcomes  in  many 
programs  once  a  program  stopped  receiving  funding,  which  consequently 
limited  visibility  into  the  extent  of  successful  transitions  within  the  DOD 
portfolio. 


Selected  Programs 
Indicate  That  Several 
Factors  Contribute  to 
Successful  Transitions 


DARPA  has  undertaken  efforts  to  understand  the  elements  that  contribute 
to  or  impede  successful  technology  transitions.  According  to  DARPA 
officials,  a  technology’s  maturity  level,  availability  of  military  service 
funding,  alignment  with  military  service  requirements,  and  transition 
planning  by  the  program  manager  influence  whether  or  not  a  DARPA- 
developed  technology  successfully  transitions.  These  characteristics  align 
with  the  findings  of  a  2001  DARPA-funded  study,  which  reported  that 
mission,  program  manager  turnover,  timing,  funding,  and  regulations, 
among  other  elements,  affect  transition  success.16 


16 Transitioning  DARPA  Technology,  Potomac  Institute  for  Policy  Studies  (Arlington,  Va.: 
May  2001). 
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In  our  review  of  10  case  study  programs,  we  found  different,  but  related, 
factors  for  transition  success  as  compared  to  the  ones  put  forward  by 
DARPA:  (1 )  military  or  commercial  demand  for  the  technology,  (2)  linkage 
to  a  research  area  where  DARPA  has  sustained  interest,  (3)  active 
collaboration  with  potential  transition  partners,  and  (4)  achievement  of 
clearly  defined  technical  goals.  Based  on  our  analyses,  we  identified  two 
factors — military  or  commercial  demand  for  the  planned  technology  and 
linkage  to  a  research  area  where  DARPA  has  sustained  interest — as 
factors  that  were  generally  evident  at  program  initiation  and  were  most 
important  to  transition.  The  remaining  two  factors — active  collaboration 
with  potential  transition  partners  and  achievement  of  clearly  defined 
technical  goals — sequentially  follow  the  first  two  factors  and  become 
observable  once  a  program  is  underway.  Figure  3  highlights  these  four 
factors. 


Figure  3:  Several  Factors  Contributed  to  Successful  Technology  Transition  in 
Selected  DARPA  Programs 
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Source:  GAO  analysis  of  Defense  Advanced  Research  Projects  Agency  (DARPA)  information.  |  GAO-16-5 


In  reviewing  the  10  programs,  we  found  that  the  existence  of  the  factors 
identified  varied  from  program  to  program.  We  assessed  the  extent  to 
which  the  four  factors  were  present  within  the  10  programs  we  reviewed, 
and  table  4  highlights  these  results.17 


17More  details  on  our  analysis  are  described  in  appendix  I. 
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Table  4:  GAO  Analysis  of  Technology  Transition  Success  Factors  and  Outcomes  within  Selected  DARPA  Programs 
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Source:  GAO  analysis  of  DOD  documentation  and  information.  |  GAO-16-5 


Military  or  Commercial 
Demand  for  the  Planned 
Technology 


We  found  that  successful  transitions  were  often  underpinned  by  existing 
military  or  commercial  demand  for  the  technology.  DARPA  officials  told  us 
that  all  of  the  agency’s  programs  are  linked  to  military  and  joint  service 
needs  at  a  high  level,  but  through  our  analyses,  we  found  that  this 
commitment  was  exemplified  when  any  of  the  following  components  were 
present  in  the  program  files: 

•  Agreement  between  DARPA  and  (1)  a  military  service,  (2)  a  DOD 
research  agency  or  laboratory,  or  (3)  other  warfighter  representative 
that  a  related  military  capability  gap  or  requirement  exists;  or 

•  A  private  company  identified  a  commercial  demand  for  the  technology 
or  showed  an  interest  in  commercializing  it. 
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For  example,  the  Spoken  Language  Communication  and  Translation 
System  for  Tactical  Use  (TRANSTAC)  program  addressed  a  known 
capability  gap  for  speech  translation  technology  within  the  Army.  As  a 
result,  the  Army  developed  the  appropriate  requirements  documents  that 
allowed  the  technology  to  successfully  transition  to  an  Army  acquisition 
program  of  record.  These  documents  identified  desired  performance 
attributes  and  system  parameters,  which  served  to  better  define  and 
communicate  the  Army’s  need  for  TRANSTAC.  The  Army’s  decision  to 
validate  specific  performance  requirements  provided  TRANSTAC  an 
opportunity  to  transition  into  an  Army  program  of  record. 

Within  our  10  case  studies,  we  found  that  a  military  or  commercial 
demand  was  fully  present  within  four  of  the  five  programs  that 
successfully  transitioned.  In  two  cases — TRANSTAC  and  Quint 
Networking  Technology  (QNT) — near-term  military  demand  was  a  result 
of  DOD’s  ongoing  involvement  in  warfighting  operations.  However,  in  the 
other  two  cases,  an  immediate  military  need  for  the  technology  was  not 
as  prevalent.  A  fifth  program  that  transitioned,  Advanced  Wireless 
Networks  for  Soldier  (AWNS),  initially  was  in  demand  by  the  Army,  but 
interest  waned  over  time  as  other  options  for  radio  networking  platforms 
emerged.  In  addition,  several  programs  developed  technologies  that 
demonstrated  military  applicability  but  lacked  a  military  or  commercial 
demand,  which  precluded  successful  transition.  For  example,  the 
Predicting  Health  and  Disease  (PHD)  and  Nastic  Materials  programs 
successfully  demonstrated  innovative  research  concepts  that  had 
potential  military  applications,  but  an  immediate  military/commercial 
demand  simply  did  not  exist  without  further  maturation  of  technologies 
past  the  point  of  program  completion. 

Linkage  to  a  Research  Area  We  also  found  that  a  program’s  linkage  to  a  research  area  in  which 

Where  DARPA  Has  Sustained  DARPA  has  sustained  interest  often  facilitated  successful  transition.  This 

Interest  interest  was  demonstrated  by  evidence  that  in  the  years  preceding  the 

program’s  initiation,  at  least  two  related  DARPA  or  other  related  DOD 
science  and  technology  programs  had  been  completed.  Sustained 
interest  is  also  exemplified  by  a  program’s  reuse  of  existing  research 
facilities  and  data  from  related  programs,  among  other  things.  Of  our  10 
case  studies,  all  5  programs  that  successfully  transitioned  were  fully 
linked  to  sustained  research  interests,  whereas  4  of  the  5  non¬ 
transitioning  programs  did  not  have  any  such  linkage. 
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Active  Collaboration  with 
Potential  Transition  Partners 


DARPA’s  program  portfolio  is  currently  organized  around  10  research 
focus  areas  under  four  key  research  themes.18  DARPA  officials  report 
that  the  Hypersonics  Capability  focus  area,  for  example,  reflects  an 
ongoing  interest  for  the  agency  that  dates  back  to  the  mid-1980s.  The 
Falcon  Combined-cycle  Engine  Technology  (FaCET)  is  one  of  several 
recent  DARPA  programs  within  the  Hypersonics  Capability  focus  area.  In 
addition,  FaCET’s  research  was  done  in  concert  with  other  hypersonic 
programs  within  DOD.  As  a  result  of  this  sustained  interest,  FaCET 
technologies  transitioned  to  other  hypersonics  programs,  including 
DARPA’s  Mode  Transition  program,  the  joint  DARPA/Air  Force 
Hypersonic  Air-breathing  Weapon  Concept  program,  and  the  Air  Force 
Research  Laboratory’s  Robust  Scramjet  and  Enhanced  Operability 
Scramjet  Technology.  Moreover,  due  to  the  National  Aeronautics  and 
Space  Administration’s  (NASA)  sustained  involvement  in  FaCET, 
technologies  were  also  transitioned  to  NASA’s  Glenn  Research  Center’s 
Combined-Cycle  Engine  Large  Scale  Inlet  Mode  Transition  Experiment 
program. 

We  found  that  in  all  five  cases  where  transition  occurred,  active 
collaboration  with  potential  transition  partners  was  fully  present.  This 
collaboration  generally  consisted  of  early  program  involvement  by 
stakeholders  within  the  government  and  commercial  sectors,  service 
requirements  officials,  and  military  liaison  officers,  among  others.  DARPA 
program  managers  were  responsible  for  facilitating  this  early  stakeholder 
involvement,  including  identifying  the  potential  transition  partners  needed 
to  assist  with  their  programs.  According  to  DARPA  officials,  achieving 
active  collaboration  with  potential  transition  partners  is  highly  dependent 
on  the  nature  of  the  program  and  background  of  the  program  manager, 
which  might  be  in  academia,  private  industry,  or  military  services.  For 
example,  a  program  manager  with  a  military  background  might  be  familiar 
with  DOD’s  acquisition  process  and  have  connections  with  service 
officials  who  can  facilitate  transition.  On  the  other  hand,  a  program 
manager  with  an  academic  background  might  lack  DOD  service 


1 R 

DARPA’s  four  research  themes  include  (1)  rethinking  complex  military  systems,  (2) 
mastering  information  at  massive  scale,  (3)  biology  as  technology,  and  (4)  new 
foundations  for  technological  surprise.  The  10  focus  areas  related  to  these  overarching 
themes  are  (1)  air  superiority  in  contested  environments;  (2)  undersea  capabilities;  (3) 
electromagnetic  spectrum  dominance;  (4)  position,  navigation,  and  timing  beyond  global 
positioning  satellites;  (5)  hypersonics  capability;  (6)  overmatch  squad;  (7)  robust  space; 
(8)  defense  against  terrorism;  (9)  cyber  capability;  and  (1 0)  big  data. 
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connections,  in  which  case  DARPA’s  military  liaison  officers  can  be  used 
to  facilitate  collaboration. 


Achievement  of  Clearly 
Defined  Technical  Goals 


DARPA’s  Architecture  for  Diode  High  Energy  Laser  Systems  (ADHELS) 
program  exemplifies  how  active  collaboration  with  potential  transition 
partners  can  facilitate  successful  technology  transition.  ADHELS 
development  included  several  technological  components,  including 
volume  bragg  grating  (VBG)  technology.  VBG  is  a  transparent  device 
made  of  refractive  glass  that  when  combined  with  a  diode  laser  can 
control  the  laser  output — such  as  by  magnifying  laser  power,  narrowing  a 
laser  beam,  or  controlling  the  beam  quality  of  the  laser  diode.  According 
to  DARPA  officials,  the  agency  contracted  with  the  foremost  experts  on 
VBG  technology  to  develop  ADHELS  components,  recognizing  that 
adaptations  of  the  VBG  technology  had  potential  applications  within  the 
commercial  marketplace.  As  ADHELS  development  progressed,  DARPA 
continued  to  engage  its  performers,  who  then  licensed  the  VBG 
technology  to  an  ADHELS  subcontractor.  This  subcontractor  formed  the 
commercial  entity  Optigrate  to  further  develop  the  VBG  technology  for 
commercial  sale. 

Conversely,  the  programs  that  lacked  active  collaboration  with  potential 
transition  partners  encountered  challenges  such  as  funding  shortfalls, 
requirements  uncertainties,  and  underperforming  technologies.  For 
example,  early  technical  challenges  prompted  DARPA  to  restructure  the 
Self-Regenerative  Systems  (SRS)  program  to  focus  exclusively  on 
technology  maturation,  canceling  initial  plans  to  demonstrate  and 
evaluate  SRS  technologies  on  a  transition  partner’s  system.  This  decision 
constrained  opportunities  to  identify  potential  transition  partners  and 
actively  collaborate  with  them  during  the  program. 

We  found  that  defining  and,  ultimately,  achieving  clear  technical  goals 
helped  facilitate  technology  transition.  Of  the  five  programs  that 
successfully  transitioned,  this  factor  was  fully  present  in  three  programs 
and  partially  present  in  the  remaining  two.  Clearly  defined  technical  goals 
often  existed  in  the  form  of  documented  agreements  among  stakeholders 
that  outlined  technical  specifications  and  desired  capabilities,  funding 
requirements,  development  schedule,  and  organizational  responsibilities 
for  technology  development.  These  agreements  allowed  DARPA  to  share 
development,  management,  and  funding  responsibilities  with  its  service 
partners,  which  facilitated  shared  understanding  of  technical  goals  and 
mutual  commitments  to  the  program’s  success  and  transition.  Equally 
important  to  this  factor  though  was  the  degree  to  which  a  program 
achieved  its  stated  technical  goals.  Most  of  the  programs  we  reviewed 


Page  16 


GAO-16-5  DARPA  Technology  Transition 


identified  clear  technical  goals,  but  fewer  than  half  actually  achieved  the 
technical  goals  that  were  originally  set. 

DARPA’s  QNT  program  represents  one  example  where  clearly  defined 
technical  goals  were  set  and  achieved.  QNT  was  initiated  with  support 
from  the  Air  Force  and  Navy,  which  helped  DARPA  craft  clear  technical 
goals  including  size,  weight,  robustness,  transmission  rates,  and  other 
performance  attributes  of  the  technology.  Defining  technical  goals  during 
the  early  stages  of  the  program  also  secured  each  organization’s 
commitment  to  playing  a  role  in  managing,  developing,  funding, 
demonstrating,  and  testing  QNT.  As  a  result,  stakeholders  then  worked 
together  to  test  QNT  technical  performance  at  several  military  exercises 
and  in  theater,  where  the  system  performed  to  expectations  and  gained 
added  exposure  within  DOD.  Ultimately,  QNT  transitioned  to  the  Army’s 
Intelligence,  Surveillance,  and  Reconnaissance  Network  program,  which 
fielded  the  system  in  Afghanistan  in  September  201 1 .  QNT  also 
transitioned  to  two  Navy  weapons  programs  and  was  also  selected  by  the 
Air  Force  for  use  in  its  Battlefield  Airborne  Communications  Node 
program,  which  hosts  a  data  link  communications  system  between  aircraft 
and  ground  units. 

In  other  cases,  such  as  AWNS,  ADHELS,  and  Nastic  Materials,  technical 
goals  were  clearly  defined,  but  only  partially  met.  These  partial 
successes,  nonetheless,  produced  substantive  technological  gains.  In  the 
cases  of  AWNS  and  ADHELS,  these  gains — coupled  with  the  presence  of 
other  key  factors — proved  sufficient  to  promote  technology  transition.  On 
the  other  hand,  three  programs  lacked  clearly  defined  goals — or  did  not 
substantively  achieve  those  goals — which  led  to  significant  restructuring 
or  development  of  technologies  that  did  not  align  with  the  needs  of  a 
planned  transition  partner.  For  example,  Marine  Corps’  officials  stated 
that  the  Tactical  Underwater  Navigation  System  relied  on  divers 
swimming  at  unsustainable  speeds  to  calibrate  its  positioning,  which  was 
not  responsive  to  their  interests. 


DARPA  Prioritizes 
Innovation  and 
Deemphasizes 
Technology  Transition 
in  Key  Processes 


DARPA’s  investment  of  program  funds  and  staff  are  primarily  focused  on 
the  highest  priority  of  its  agency  mission,  which  is  creating  radically 
innovative  technologies  that  support  DOD’s  warfighting  mission. 
Technology  transition  is  a  secondary  priority  at  the  agency.  DARPA 
leadership  conducts  periodic  reviews  of  agency  programs,  but  these 
reviews  are  focused  on  scientific  and  technical  aspects  of  the  programs 
and  do  not  assess  technology  transition  strategies.  Instead,  the  Director, 
DARPA,  delegates  responsibility  for  oversight  and  assessment  of 
technology  transition  strategies  to  a  subordinate  office.  DARPA  also 
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provides  limited  training  to  program  managers  related  to  technology 
transition,  instead  relying  on  others  within  the  agency  to  assist  program 
managers  with  this  activity,  as  needed.  In  addition,  although  DARPA 
disseminates  information  on  its  past  programs,  it  does  not  take  full 
advantage  of  available,  government-sponsored  resources  for  sharing 
technical  data.  DARPA  has  also  elected  not  to  participate  in  most  DOD 
programs  intended  to  facilitate  technology  transition,  with  the  exception  of 
mandated  small  business  programs,  citing  the  challenges  it  perceives  in 
meeting  the  process  and  reporting  requirements  of  these  DOD  programs 
within  DARPA’s  typical  timeframes  for  executing  its  research  initiatives. 


DARPA  Investments  in 
People  and  Programs 
Drive  a  Culture  of 
Innovation 


At  DARPA,  the  desire  for  innovation  drives  investment,  both  in  terms  of 
recruitment  and  programs.  DARPA  hires  world-class  scientists  and 
engineers  from  private  industry,  universities,  government  laboratories, 
and  research  centers  to  serve  as  program  managers.  According  to 
DARPA  officials,  program  managers  are  given  great  flexibility  in  leading 
their  programs,  building  their  teams,  and  allocating  funds  to  achieve  their 
programs’  objectives,  including  technology  transition.  DARPA  officials 
stated  that  these  expectations  are  outlined  to  program  managers  during 
new  hire  orientations,  but  are  not  codified  in  any  agency-wide  policy  or 
guidance.  To  ensure  that  new  ideas  for  advanced  technologies  are 
continuously  coming  into  DARPA,  the  agency  usually  limits  the  tenure  of 
its  program  managers,  as  well  as  the  duration  of  its  programs,  to  3  to  5 
years.  In  this  environment,  program  managers  prioritize  achieving 
programs’  technical  objectives,  which  can  require  the  overwhelming 
majority  of  their  available  time. 

This  focus  on  innovation,  which  corresponds  with  undertaking  bold, 
ambitious  programs,  makes  the  pursuit  of  technology  transition  a 
secondary  priority  for  the  agency.  Consequently,  programs  generally  seek 
to  prove  the  art  of  “what  is  possible”  rather  than  refining,  producing,  and 
delivering  tactical  equipment  to  warfighters.  According  to  DARPA  officials, 
the  agency  views  these  latter  processes  as  the  responsibility  of  military 
service  research  agencies,  laboratories,  and  acquisition  programs  of 
record.  However,  DARPA  officials  report  that  potential  transition  partners 
in  the  acquisition  community  are  often  unwilling  to  commit  to  incorporating 
new  technology  into  their  programs  of  record  without  additional 
maturation,  and  service  research  agencies  and  laboratories  both  have 
their  own  programs  and  priorities  to  pursue.  Consequently,  the  additional 
maturation  work  needed  to  position  DARPA  programs  for  effective 
transitions  can  go  unfunded.  According  to  DARPA  officials,  this  dynamic 
has  proven  to  be  a  major  impediment  for  the  agency  in  transitioning 
technology. 
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In  addition,  the  introduction  of  DARPA’s  radically  innovative  technologies 
can  disrupt  the  status  quo  for  military  programs,  budgets,  and  warfighting 
doctrine,  which  can  drive  cultural  opposition  within  the  military  services. 
DARPA  officials  stated  that  the  agency’s  research  sometimes  leads  to  the 
identification  of  technologies  and  capabilities  that  military  service  officials 
do  not  initially  want  or  think  their  services  will  need,  although  these 
technologies  can  eventually  provide  important  military  capabilities.  For 
example,  DARPA  officials  said  that  the  Air  Force  was  initially  highly 
resistant  to  investments  in  stealth  technologies  for  aircraft.  Despite  this 
resistance,  DARPA  proceeded  with  the  development  of  stealth 
technologies,  and  today  they  are  in  use  on  multiple  DOD  weapon 
systems,  including  the  F-22  Raptor  and  F-35  Lightning  II  fighter  aircraft. 

DARPA’s  secondary  emphasis  on  transition  is  a  long-standing 
characteristic  of  the  agency’s  culture,  as  evidenced  in  studies 
commissioned  by  DARPA  in  1985  and  2001,  which  found  that  the  agency 
does  not  place  enough  emphasis  on  technology  transition.19  The  1985 
report  recommended  that  DARPA  designate  full-time  technology 
transition  facilitators,  due  to  problems  that  were  identified  in  the  transition 
of  technologies  to  the  military  services.  The  2001  report  recommended 
matching  program  manager  tenure  to  the  expected  length  of  the 
programs  to  which  they  are  assigned — rather  than  setting  arbitrary  dates 
of  departure — and  defining  additional  training  and  incentives  for 
technology  transition. 

According  to  DARPA  officials,  the  Director,  DARPA,  has  undertaken 
several  initiatives  to  improve  the  agency’s  emphasis  on  technology 
transition,  including  transition-focused  quarterly  meetings  with  each  of  the 
military  service  chiefs  or  their  deputies  and  establishment  of  the  Adaptive 
Execution  Office  in  2013,  which  was  chartered  to  accelerate  the  transition 
of  game-changing  DARPA  technologies  into  DOD  capabilities.  In  addition, 
DARPA  officials  stated  that  the  Director  has  shifted  the  role  of  the 
agency’s  military  service  liaisons  to  focus  exclusively  on  assisting 
program  managers  with  military  service  engagement  and  transition  of 
DARPA  technologies.  DARPA  officials  report  that  these  actions  have 
elevated  the  priority  of  and  resources  devoted  to  technology  transition 
within  the  agency. 


1 Q 

Technology  Transfer  at  DARPA,  Technology  Transfer  Center,  George  Mason  University 
(Fairfax,  V a.:  Dec.  1985)  and  Transitioning  DARPA  Technology,  Potomac  Institute  for 
Policy  Studies  (Arlington,  Va.:  May  2001 ). 
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DARPA  Leadership 
Forgoes  Opportunities  to 
Assess,  and  Thus 
Potentially  Improve, 
Technology  Transition 
Strategies 


The  Director,  DARPA,  conducts  oversight  of  programs  through  periodic 
milestone  reviews.  These  reviews  assess  a  program’s  scientific  and 
technical  merit,  and,  according  to  DARPA  officials,  provide  the  Director 
with  information  on  the  transition  status  of  the  program.  According  to 
DARPA  officials,  the  scope  of  these  reviews  is  reflective  of  and  consistent 
with  the  agency’s  top  priority  of  creating  innovative  technologies. 

However,  these  reviews  do  not  assess  a  program’s  strategy  for  achieving 
technology  transition.  DARPA  leadership  delegates  oversight  and  review 
of  technology  transition  strategies  to  the  agency’s  Adaptive  Execution 
Office,  which  coordinates  with  program  managers  to  review  and  provide 
input  on  technology  transition  strategies,  particularly  in  the  latter  stages  of 
programs. 


DOD  policy,  however,  assigns  to  the  Director,  DARPA,  the  responsibility 
to  pursue  “strategies”  that  “increase  the  impact  of  DARPA’s  research  and 
development  programs”  and  “speed  the  transition  of  successful  research 
and  development  programs  to  the  military  departments  and  defense 
agencies,”  among  other  scientific  and  technical  functions.20 
Consequently,  by  not  assessing  technology  transition  strategies  at  the 
program  milestone  reviews  it  chairs,  the  Director,  DARPA,  is  forgoing  key 
opportunities  to  perform  this  function.  This  approach  undermines 
transition  planning  and  introduces  risk  that  DARPA  programs  will  not 
achieve  their  full  transition  potential. 


Apart  from  the  policy  cited  above,  the  Office  of  the  Secretary  of  Defense 
does  not  maintain  other  instructions  or  directives  related  to  technology 
transition  at  DARPA.  In  previous  years,  different  components  within  the 
Office  of  the  Secretary  of  Defense  have  issued  nonmandatory  guidance 
on  technology  transition,  which  has,  at  times,  applied  to  DARPA 
programs.21  However,  the  guidance  is  now  outmoded  in  that  it  does  not 
address  changes  in  key  science,  technology,  and  acquisition  processes 
that  have  occurred  during  the  last  10  years.  The  Office  of  the  Assistant 


20DOD  Directive  5134.10,  Subject:  Defense  Advanced  Research  Projects  Agency 
(DARPA)  (May  7,  2013). 

21Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics, 
Defense  Procurement  and  Acquisition  Policy,  Manager’s  Guide  to  Technology  Transition 
in  an  Evolutionary  Acquisition  Environment ,  Version  1.0  (Washington,  D.C.:  Jan.  31, 2003) 
and  Defense  Acquisition  University  and  Office  of  the  Deputy  Under  Secretary  of  Defense 
for  Advanced  Systems  and  Concepts,  Manager’s  Guide  to  Technology  Transition  in  an 
Evolutionary  Acquisition  Environment ,  Version  2.0  (DOD/DAU  University  Press,  Fort 
Belvoir,  Va.:  June  2005). 
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Secretary  of  Defense  for  Research  and  Engineering,  which  has  primary 
oversight  of  DARPA  and  other  DOD  research  agencies,  provides  a  great 
deal  of  latitude  to  these  agencies  to  define  their  own  technology  transition 
policies  and  procedures.  Most  notably,  officials  from  this  office  stated  to 
us  that  technology  transition  is  no  longer  an  explicit  function  of  the  office 
and  that  the  DOD  division  formerly  responsible  for  technology  transition 
no  longer  exists.  Instead,  the  office  now  limits  its  technology  policy 
responsibilities  to  minimizing  unnecessary  duplication  of  research  efforts 
within  DOD,  disseminating  research  knowledge  throughout  DOD,  and 
sharing  that  information  with  the  general  public.22 


Current  Training  for 
Managers  Does  Not 
Consistently  Position 
Programs  for  Transition 
Success 


DARPA’s  program  managers  receive  limited  training  on  how  to  effect 
technology  transition  in  their  programs.  This  training  consists  primarily  of 
overviews  on  DARPA’s  technology  transition  paths  and  considerations  to 
make  at  program  milestones  with  respect  to  technology  transition. 

DARPA  program  managers  are  not  subject  to  the  formal  training  and 
certification  requirements  applicable  to  permanently  hired  science  and 
technology  managers  at  military  service  laboratories.  DOD  requires 
managers  in  these  laboratories  to  complete  Defense  Acquisition 
University  (DAU)  training  courses  in  science  and  technology,  including 
how  they  apply  to  technology  transition.  These  courses  lead  to 
progressively  higher  knowledge  and  certifications  over  their  careers. 
DARPA  officials  countered  that  the  training  necessary  to  complete  these 
courses  and  achieve  science  and  technology  manager  certifications 
would  require  DARPA  program  managers  to  devote  an  inordinate  amount 
of  time  to  training,  particularly  if  DAU  requires  DARPA  staff  to  complete 
all  the  typical  prerequisite  courses  that  other  managers  are  required  to 
complete.  Further,  given  the  agency’s  unique  mission,  DARPA  officials 
stated  that  they  do  not  consider  the  DAU  training  courses  to  be  as 
relevant  to  their  program  managers  given  the  agency’s  broad  discretion  to 
pursue  breakthrough  technologies  versus  specific  management  of 
acquisition  programs. 

In  lieu  of  more  robust  training,  DARPA  officials  stated  that  the  agency 
supports  its  program  managers’  transition  efforts  by  providing  them  with 
access  to  various  transition  planning  and  outreach  resources.  For 
example,  DARPA  program  managers  are  also  supported  by  the  agency’s 


22These  policies  include  DOD  Instruction  3200.12,  Subject:  DOD  Scientific  and  Technical 
Information  Program  (Aug.  22,  2013);  and  DOD  Instruction  5535.8,  Subject:  DOD 
Technology  Transfer  (T2)  Program  (May  14,  1999). 
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Adaptive  Execution  Office,  which  provides  them  with  assistance  in 
developing  their  transition  plans  and  in  communicating  with  the  military’s 
transition  stakeholders  at  DOD’s  combatant  commands.  Program 
managers  are  further  supported  by  DARPA’s  military  liaison  officers  from 
the  Air  Force,  Army,  Navy,  and  Marine  Corps,  who  help  program 
managers  identify  and  reach  out  to  potential  transition  partners  or  other 
stakeholders  in  the  military  services.  These  liaisons  also  arrange  for 
DARPA  leadership  to  meet  with  the  military’s  senior  leaders,  which  is 
done  in  part  to  advocate  for  the  transition  of  DARPA  programs  to  the 
military  services.  As  we  found  in  September  2006,  these  liaisons  can  also 
provide  operational  advice  for  planning  and  strategy  development  and 
provide  an  understanding  of  service  perspectives,  issues,  and  needs  so 
that  potential  customers  can  be  identified  and  effective  agreements  can 
be  written.23  Program  managers  also  are  authorized  to  use  program  funds 
to  hire  experienced  contractors  and  government  staff  from  other  agencies 
to  aid  technology  transition  activities  in  their  programs. 

Previous  guidance  and  studies,  including  one  commissioned  by  DARPA 
in  2001 ,  have  recommended  that  DARPA  improve  its  technology 
transition  training  for  program  managers  through  additional  training  and 
mentoring  programs  related  to  technology  transition.24  Further,  in  2005, 
DOD  issued  guidance  on  technology  transition  stating  that  developing 
and  executing  a  training  plan  for  the  members  of  the  team  supporting 
technology  transition  is  essential  to  their  success.25  Similarly,  Standards 
for  Internal  Control  in  the  Federal  Government  indicates  that  effective 
management  of  an  organization’s  workforce,  which  includes  providing 
necessary  training  to  the  organization’s  staff,  is  essential  to  achieving 
results.26  DARPA’s  limited  training  for  program  managers  on  technology 
transition  is  inadequate  to  consistently  position  programs  for  transition 
success.  Without  sufficient  training,  program  managers  may  not  develop 


23GAO-06-883. 

24Transitioning  DARPA  Technology,  Potomac  Institute  for  Policy  Studies  (Arlington,  Va.: 
May  2001). 

25DAU  and  the  Office  of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems 
and  Concepts,  Manager’s  Guide  to  Technology  Transition  in  an  Evolutionary  Acquisition 
Environment,  Version  2.0  (Fort  Belvoir,  Va.:  June  2005). 

26GAO,  Internal  Control:  Standards  for  Internal  Control  in  the  Federal  Government, 
GAO/AIMD-OO-21 .3.1  (Washington,  D.C.:  November  1999). 
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the  skills  and  knowledge  that  they  need  to  identify  and  engage  potential 
transition  partners  and  facilitate  transition  successes. 

While  DARPA  does  not  currently  rely  on  other  DOD  entities  for 
technology  transition  training,  individual  DARPA  program  managers  may 
voluntarily  elect  to  take  training  related  to  technology  transition  in  DOD  or 
other  federal  organizations.  For  example,  the  Federal  Laboratory 
Consortium  for  Technology  Transfer  was  established  by  law  in  the 
Federal  T echnology  T ransfer  Act  of  1 986  to,  among  other  things,  (1 ) 
develop  training  for  federal  lab  employees  engaged  in  technology  transfer 
and  (2)  facilitate  communication  and  cooperation  between  federal 
laboratories.27  The  Consortium  offers  both  in-person  and  online  training 
regarding  commercialization  of  technologies,  as  well  as  guidance 
regarding  best  practices.  In  response  to  our  inquiries  on  this  subject, 
DARPA  officials  indicated  they  were  in  discussions  with  the  DAU  staff 
regarding  potential  future  training  options. 


Shortfalls  in  Disseminating 
Program  Information  May 
Lead  to  Missed  Transition 
Opportunities 


Disseminating  information  regarding  developed  technologies  is  a  way  for 
agencies  to  promote  technology  transition  after  the  conclusion  of  a 
program,  particularly  once  program  managers  and  staff  are  no  longer 
actively  advocating  for  the  transition  of  their  program’s  technologies.  For 
many  years,  DOD  has  maintained  website-accessible  databases  that 
disseminate  information  within  the  department,  and  to  a  lesser  extent,  to 
the  public  and  to  private  companies.  These  websites  allow  their  users  to 
search  for  related  technologies  while  considering  new  programs  or 
products  that  could  possibly  use  them. 


While  DARPA  disseminates  information  on  past  programs  through  the 
use  of  public  government  websites,  its  selective  approach  to  posting  this 
information  does  not  maximize  the  chances  of  DARPA  technologies  being 
identified  and  selected  by  potential  transition  partners.  Currently,  DARPA 
disseminates  information  on  past  programs  through  both  internal  and 
external  means,  but  does  not  share  information  with  key  data  repositories 
that  the  federal  government  sponsors,  which  may  obscure  visibility  into  its 
programs  and  lead  to  missed  transition  opportunities.  Since  the  1960s, 
DARPA  has  provided  substantial  amounts  of  information  regarding  its 
technologies  to  the  official  DOD  dissemination  website  managed  by  the 
Defense  Technical  Information  Center  (DTIC).  Although  the  majority  of 


27Pub.  L.  No.  99-502. 
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DARPA-related  information  in  this  database  is  restricted  to  DOD  staff,  it  is 
by  far  the  largest  repository  that  private  companies  and  the  general  public 
can  access  for  information  on  DARPA  technologies.  For  instance,  we 
found  that  while  non-DOD  users  can  access  approximately  3,600  DARPA 
technical  records  through  DTIC’s  public  website,  DOD  users  can  access 
over  30,000  of  these  records.  DARPA  also  maintains  an  “Open  Catalog” 
public  website  for  disseminating  information  on  its  programs,  although  it 
currently  only  has  technical  information  on  about  a  few  dozen  active  and 
completed  programs.  In  comparison,  DARPA’s  public  website  also 
provides  brief,  non-technical  descriptions  of  194  active  DARPA  programs. 
Two  other  government-sponsored  websites,  operated  by  the  DOD 
TechLink  public-private  partnership  and  the  Federal  Laboratory 
Consortium  for  Technology  Transfer,  also  exist  to  help  science  and 
technology  agencies  disseminate  technology  information.  DARPA  officials 
indicated  they  do  not  share  information  with  either  of  these  entities  and 
instead  exclusively  rely  on  DTIC,  which  DARPA  officials  stated 
represented  DOD’s  official  repository. 

In  recent  years,  the  White  House  has  provided  direction  to  broaden 
access  to  non-sensitive  information  on  government-developed 
technologies,  in  recognition  of  government  research’s  potential  for 
catalyzing  innovative  breakthroughs  that  drive  the  U.S.  economy,  and 
helping  to  drive  progress  in  areas  such  as  health,  energy,  the 
environment,  agriculture,  and  national  security.  In  February  2013,  the 
White  House’s  Office  of  Science  and  Technology  Policy  instructed  the 
federal  government’s  science  and  technology  community  to  begin 
planning  how  to  disseminate  information  on  technologies  they  have 
developed.28  According  to  DARPA  officials,  the  lead  DOD  agency  for 
implementing  this  system  is  DTIC,  and  they  do  not  expect  DOD  to  have  a 
dissemination  system  in  place  that  fully  addresses  the  requirements  of 
the  memorandum  until  2017. 


oo 

Executive  Office  of  the  President,  Office  of  Science  and  Technology  Policy,  Subject: 
Increasing  Access  to  the  Results  of  Federally  Funded  Scientific  Research  (Washington, 
D.C.:  Feb.  22,  2013). 
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DARPA  Participates  in 
Mandated  Small  Business 
Programs,  but  Refrains 
from  Other  DOD-Wide 
Programs  Intended  to 
Facilitate  Technology 
Transition 


The  Office  of  the  Secretary  of  Defense  manages  several  DOD  programs 
intended  to  accelerate  development,  testing,  and  delivery  of  mature 
technologies  that  provide  new  solutions  for  military  needs.  The  general 
purpose  of  these  programs  is  to  facilitate  the  transition  of  technologies, 
but  vary  in  terms  of  what  types  of  technology  developers  and  operational 
needs  they  target.  For  example,  in  partnership  with  the  military  services, 
the  Joint  Capability  Technology  Demonstration  program  addresses  joint 
warfighting  needs  of  the  combatant  commands  by  demonstrating  mature 
technology  prototypes  that  may  transition  to  acquisition  programs  or 
directly  to  the  warfighter  in  the  field.  Other  programs  such  as  the  Small 
Business  Innovation  Research  (SBIR)  program  fund  small  business 
research  and  development  with  the  goal  that  innovations  produced  will  be 
commercialized  and  eventually  sold  back  to  DOD.  Table  5  lists  these 
programs. 


Table  5:  Overview  of  DOD  Technology  Transition  Programs 


Program 

Year 

established 

Typical 

funding 

Time  Frame  for 
completion 

Description 

Small  Business 
Innovation  Research 
(SBIR)  and  Small 
Business  Technology 
Transfer  (STTR) 

1982  and  1992 

$1 .65  million 
or  less 

42  months 
or  less 

Government-wide,  legally  mandated  programs  coordinated 
with  the  Small  Business  Administration,  in  which  all  federal 
agencies  with  research  budgets  in  excess  of  $100  million 
(SBIR)  and  $1  billion  (STTR)  must  reserve  about  3  percent 
of  contracts  or  grants  for  award  to  small  businesses. 

Joint  Capability 
Technology 
Demonstration  (JCTD) 

1994 

Varies 

12-36 

months 

Facilitates  fast  and  cost-effective  technology  transition  to 
end-users  in  the  U.S.  military.  JCTDs  assess  the  military 
utility  of  new  technologies  and  facilitate  their  transition. 

Manufacturing 

Technology 

1956 

Varies 

Varies 

Develops  technologies  that  can  improve  manufacturing 
capabilities,  processes  or  costs,  or  that  may  improve 
sustainment  practices. 

Quick  Reaction  Fund 

2002 

$2.5  million- 
$3.0  million 

12  months 
or  less 

Funds  high  priority  and  short  duration  technology 
development  efforts  in  response  to  new  threats  and  to  help 
meet  the  urgent  needs  of  conventional  military  forces. 

Rapid  Reaction  Fund 

2004 

$0.5  million 

6-18 

months 

Identifies  and  develops  near-term  military  capabilities  to 
support  irregular  warfare  needs. 

Rapid  Innovation 
Program 

2011 

$3.0  million 
or  less 

24  months 
or  less 

Accelerates  the  transition  of  technologies  developed  by 
small  businesses  participating  in  SBIR  projects,  other 
businesses  and  defense  laboratories;  primarily  supports 
major  defense  acquisition  programs,  or  acquisition 
programs  meeting  a  critical  national  security  need. 

Source:  GAO  analysis  of  DOD  data.  |  GAO-16-5 

According  to  DARPA  officials,  the  only  programs  that  DARPA  participates 
in  are  the  SBIR  and  Small  Business  Technology  Transfer  (STTR) 
programs  because  it  is  legally  required  to  do  so.  However,  the  agency’s 
knowledge  of  transition  outcomes  associated  with  SBIR  and  STTR 
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expenditures  is  limited.  DARPA  officials  said  they  do  not  maintain  a 
comprehensive  list  of  agency  programs  using  SBIR  or  STTR  funds — or 
the  transition  paths  of  technologies  developed  with  these  funds — because 
these  data  are  not  always  reported  or  accessible.  According  to  DOD 
small  business  program  management  officials,  who  oversee  the  use  of 
these  funds  throughout  the  department,  transition  outcome  data  is  not 
required  from  any  military  service  or  agency,  including  DARPA.  These 
officials  further  stated  that  once  a  small  business  contract  ends,  DOD’s 
means  for  compelling  contractors  to  identify  and  report  on  successful 
transitions  expires.  In  December  2013,  we  recommended  that  DOD 
improve  its  tracking  of  technology  transition  outcomes  in  SBIR-funded 
programs  by  establishing  a  common  definition  of  technology  transition  for 
all  SBIR  projects  and  improving  the  completeness,  quality,  and  reliability 
of  SBIR  transition  data  that  it  reports.29  These  tracking  shortfalls 
precluded  us  from  assessing  the  extent  to  which  DARPA’s  SBIR  and 
STTR  funds  contribute  to  successful  technology  transitions.  In  lieu  of 
comprehensive  transition  data,  DARPA  officials  have  worked  with  some 
of  their  prior  program  contractors — who  successfully  developed  and 
transitioned  technologies — to  identify  small  business  program  success 
stories.30  In  addition,  DARPA  officials  stated  that  they  are  developing 
contract  language  for  future  SBIR  awards  that  would  require  firms  to 
identify  their  transition  and  commercialization  outcomes  as  an  addendum 
to  their  final  report. 

Apart  from  the  legally  required  small  business  programs,  DARPA  officials 
said  that  the  processes  and  reporting  requirements  associated  with 
participating  in  DOD’s  other  technology  transition  programs  are  generally 
cumbersome  and  do  not  align  with  DARPA’s  time  frames  for  executing 
programs  or  mission  of  creating  disruptive  technologies  over  relatively 
long  periods  of  time.  Conversely,  DOD  transition  programs  are  mainly 
intended  for  mature  technologies,  or  short-term  efforts  that  can  be  fielded 
quickly.  DARPA  officials  explained  that  technologies  their  programs 


29GAO-14-96. 

oq 

Among  the  successes  DARPA  cites  are  its  (1)  Micro  Air  Vehicle,  which  created  a  small 
unmanned  aircraft  capable  of  flying  up  to  a  mile  away,  and  whose  components  became 
enabling  technologies  for  the  hand-launched,  Raven  unmanned  aerial  vehicle  that  DOD 
has  employed  in  operations  around  the  world  and  (2)  MeerCAT  visualization  tool,  which 
helps  security  teams  discover,  visualize,  analyze,  and  report  wireless  threats  across 
various  locations  and  time  periods  by  showing  two-  and  three-dimensional  visualizations 
of  these  threats  on  a  computer  display. 
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develop  usually  require  additional  maturation  in  subsequent  technology 
development  efforts,  either  within  DARPA  or  at  military  service 
laboratories,  before  transitioning  to  acquisition  programs  or  warfighters. 
DARPA  officials  also  said  that  agency  leadership  generally  views  the  use 
of  these  funds  as  unnecessary  given  that  DARPA’s  budget  currently 
provides  adequate  funding  to  support  its  research  endeavors. 

DARPA  officials  also  indicated  that  they  are  exploring  stronger 
relationships  with  the  Joint  Capability  Technology  Demonstration  (JCTD) 
program,  particularly  in  the  area  of  prototyping.  In  previous  decades, 
DARPA  used  funds  from  the  predecessor  to  the  JCTD  program — then 
known  as  the  Advanced  Concept  Technology  Demonstration  program — to 
develop  and  demonstrate  technologies.  These  efforts  include  currently 
fielded  systems  such  as  the  Air  Force’s  Global  Hawk  and  Predator 
unmanned  aircraft  and  Miniature  Air-Launched  Decoy  systems.  DARPA 
officials  also  stated  that  Manufacturing  Technology  program  funds  have 
been  applied  after  DARPA  program  completions  to  improve  the 
affordability  of  and  manufacturing  base  for  semiconductors  developed  by 
DARPA. 


Conclusions 


Technology  transition  does  not  have  to  occur  at  the  expense  of 
innovation,  but  should  instead  be  viewed  as  a  natural  extension  of 
innovation.  When  DARPA  places  technology  in  the  hands  of  a  user, 
operational  knowledge  is  gained  that  can  be  used  to  improve  the 
technology  and  further  scientific  innovation.  However,  DARPA  leadership 
does  not  fully  subscribe  to  this  viewpoint;  instead,  it  is  satisfied  with 
maturing  technology  to  the  point  where  feasibility,  but  not  functionality,  is 
proven.  Today,  programs  progress  through  DARPA  without  the  agency 
head  fully  assessing  whether  transition  strategies  make  sense.  Such 
assessments,  if  measured  against  key  transition  factors,  could  improve  a 
program’s  potential  for  transition  success.  Transition  responsibilities  then 
fall  almost  exclusively  on  individual  program  managers,  who  are  often  not 
sufficiently  trained  to  achieve  the  favorable  transition  outcomes  they  seek. 
Further,  when  the  program  manager’s  tenure  expires,  the  primary 
advocate  for  transitioning  the  program’s  technology  is  also  lost.  This 
turnover  increases  the  need  for  technical  gains  to  be  appropriately 
documented  and  disseminated  so  that  user  communities  have  visibility 
into  potential  solutions  available  to  meet  their  emerging  needs.  An 
important  part  of  this  process  is  the  tracking  of  transition  outcomes,  as  we 
recommended  DOD  undertake  for  its  technology  transition  programs  in 
March  2013,  and  which  we  have  also  found  lacking  at  DARPA. 
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Recommendations  for  'mProve  technology  transition  planning  and  outcomes  at  DARPA,  we 

recommend  the  Secretary  of  Defense  direct  the  Director,  DARPA,  to  take 
Executive  Action  the  following  three  actions: 

•  Oversee  assessments  of  technology  transition  strategies  for  new  and 
existing  DARPA  programs  as  part  of  existing  milestone  reviews  used 
to  assess  scientific  and  technical  progress  to  inform  transition 
planning  and  program  changes,  as  necessary.  Our  analysis  identified 
four  factors  that  could  underpin  these  assessments,  but  the 
uniqueness  of  individual  DARPA  programs  suggests  that  other 
considerations  may  also  be  warranted. 

•  Increase  technology  transition  training  requirements  and  offerings  for 
DARPA  program  managers,  leveraging  existing  DOD  science  and 
technology  training  curricula,  as  appropriate. 

•  Increase  the  dissemination  of  technical  data  on  completed  DARPA 
programs  through  Open  Catalog  and  other  government-sponsored 
information  repositories  aimed  at  facilitating  commercialization  of 
technologies. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  for  review  and  comment.  In  its 
written  comments,  which  are  included  in  appendix  II,  DOD  partially 
agreed  with  our  recommendations  to  oversee  assessments  of  technology 
transition  strategies  for  DARPA  programs  and  to  increase  technology 
transition  training  requirements  and  offerings  for  DARPA  program 
managers.  In  doing  so,  DOD  agreed  with  most  of  the  principles  contained 
in  our  recommendations,  but  disagreed  with  the  actions  we 
recommended.  DOD  did  not  agree  with  our  recommendation  to  increase 
the  dissemination  of  technical  data  on  completed  DARPA  programs.  DOD 
also  separately  provided  technical  comments,  which  we  incorporated,  as 
appropriate. 

DOD  agreed  that  assessments  of  technology  transition  strategies,  which 
consider  the  four  factors  we  identified  for  transition  success,  would  help 
inform  program  decisions  by  DARPA  leadership.  However,  DOD  did  not 
agree  that  such  assessments  be  required  at  milestone  reviews  for 
DARPA  programs,  citing  active  participation  by  the  Director,  DARPA,  in 
technology  transition  discussions  throughout  the  life  of  a  program.  We 
agree  that  leadership  is  focused  on  technology  transition  and  holds 
discussions  often;  however,  we  found  it  difficult  to  be  able  to  identify 
transitions — or  changes  to  transition  strategies — that  arise  from  these 
discussions.  We  believe  that  these  discussions  are  an  inadequate 
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substitute  for  assessing  technology  transition  strategies  as  part  of  the 
comprehensive  program  reviews  that  DARPA  already  undertakes. 
Assessing  transition  strategies  at  these  reviews,  as  we  recommended, 
would  provide  the  opportunity  to  coordinate  and  prioritize  transition  goals, 
objectives,  and  planned  actions  in  the  context  of  scientific  and  technical 
developments  in  the  program.  By  overseeing  technology  transition 
strategies  separate  from  these  reviews,  the  Director,  DARPA,  risks 
making  decisions  related  to  a  program’s  transition  that  are  not 
appropriately  informed  by  other  important  program  considerations. 
Although  DARPA  asserted  that  our  recommendation  runs  counter  to  its 
current  efforts  to  improve  processes  and  procedures,  we  found  no 
evidence  that  processes  and  procedures  were  improving. 

DOD  also  agreed  that  technology  transition  training  improves  transition 
planning  and  outcomes,  citing  DOD  science  and  technology  training 
curricula  as  a  “rich  repository  of  transition  insight.”  Yet,  despite  the  value 
it  sees  in  its  own  training  resources,  DOD  stated  that  DARPA  program 
managers’  relatively  short  tenure  leaves  few  opportunities  to  expose  them 
to  such  “generic”  training  opportunities.  Consequently,  DOD  did  not  agree 
that  technology  transition  training  requirements  should  be  increased  for 
DARPA  program  managers  and  stated  that  DARPA’s  current  approach  of 
“tailored  curricula  focused  on  a  program’s  unique  transition  needs” 
remained  appropriate.  However,  we  did  not  find  evidence  of  such  tailored 
curricula  in  our  review.  Instead,  we  found  that  DARPA  program  managers 
all  received  the  same  limited  training  upon  hiring,  which  was  inadequate 
to  consistently  position  programs  for  transition  success.  DOD  also  stated 
that  DARPA  continues  to  explore  opportunities  to  offer  tailored,  concise, 
and  streamlined  training  to  its  program  managers.  Therefore,  we  stand  by 
our  recommendation  and  continue  to  believe  that  expanded  training 
opportunities  are  necessary  for  achieving  better  transition  outcomes  in 
DARPA  programs,  and  we  encourage  DOD  to  capitalize  on  its  existing 
investments  in  this  area,  to  the  extent  possible. 

Further,  DOD  did  not  agree  that  increased  dissemination  of  technical  data 
on  completed  DARPA  programs  was  warranted.  DOD  stated  that  using 
multiple  information  repositories  “thins  the  DOD  technology  market  by 
spreading  it  across  several  venues,”  in  turn  reducing  the  likelihood  that 
technology  providers  and  potential  transition  partners  will  find  a  match. 
DOD  also  stated  that  it  intends  to  make  DTIC  the  central  data  storage  for 
all  DOD  technical  activities,  including  DARPA  technologies,  and  views  the 
use  of  multiple  information  repositories  as  unconducive  to  improving 
technology  transition  outcomes.  In  our  review,  we  found  DARPA’s 
existing  reliance  on  DTIC  limited  the  chances  of  the  agency’s 
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technologies  being  identified  and  selected  by  potential  transition  partners, 
particularly  those  outside  of  DOD.  We  fail  to  see  how  increased 
dissemination  of  technical  data  would  actually  “thin”  the  DOD  technology 
market.  To  the  contrary,  it  would  allow  more  portals  with  which  to  gain 
access.  Similarly,  in  2013,  the  White  House  identified  a  government-wide 
need  to  broaden  access  to  non-sensitive  information  on  government- 
developed  technologies,  but  improvements  remain  incomplete. 
Consequently,  we  continue  to  believe  that  DARPA  should  pursue 
dissemination  of  non-sensitive  technical  data  through  as  many  existing 
government-sponsored  outlets  as  possible,  including  its  own  Open 
Catalog  website  and  DOD  TechLink,  to  improve  the  likelihood  of  transition 
successes  in  the  agency’s  programs. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Secretary  of  Defense,  and  the  Director,  DARPA.  In 
addition,  the  report  is  available  at  no  charge  on  the  GAO  website  at 

http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)  512-4841  orsullivanm@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  GAO  staff  who  made  key  contributions  to  this 
report  are  listed  in  appendix  III. 


Michael  J.  Sullivan 
Director 

Acquisition  and  Sourcing  Management 
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The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Thad  Cochran 
Chairman 

The  Honorable  Richard  Durbin 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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This  report  covers  the  Defense  Advanced  Research  Projects  Agency’s 
(DARPA)  (1)  effectiveness  at  transitioning  technologies  since  fiscal  year 
2010,  including  identifying  the  factors  that  contributed  to  successful 
technology  transitions,  and  (2)  implementation  of  Department  of  Defense 
(DOD)  policies  and  programs  intended  to  facilitate  the  transition  of 
technologies. 

To  assess  DARPA’s  effectiveness  at  transitioning  technologies  since 
fiscal  year  2010,  including  identifying  factors  that  contributed  to 
successful  transitions,  we  requested  and  reviewed  portfolio-level  data 
identifying  the  names,  funding  amounts,  and  technology  transition  of 
those  DARPA  programs  successfully  completing  technology  development 
during  fiscal  years  2010  through  2014.  We  confined  our  analysis  to  this 
time  frame  owing  to  availability  of  data  from  DARPA.  These  data  included 
150  programs  funded  under  DOD’s  budget  activities  for  (1)  applied 
research  and  (2)  advanced  technology  development  that  DARPA 
identified  as  having  completed  as  planned  and  producing  a  substantive 
technological  gain  or  innovation,  regardless  of  whether  that  technology  or 
innovation  transitioned  to  an  end  user.  We  used  these  data  as  the  basis 
for  selecting  a  simple  random  sample  of  10  case  study  programs — 5  that 
transitioned  and  5  that  did  not  transition.  In  conducting  our  case  study 
analyses,  we  reviewed  relevant  program  documentation  to  identify  factors 
that  facilitate  transition  success.  While  reviewing  these  case  study 
programs,  we  identified  inconsistencies  between  agency  portfolio-level 
transition  outcome  data  and  program-level  information.  DARPA  officials 
stated  to  us  that  this  was  due  to  the  transition  status  of  these  programs 
changing  after  they  had  collected  the  portfolio-level  data.  As  a  result,  we 
concluded  that  DARPA’s  portfolio-level  data  were  not  sufficiently  reliable 
for  the  purposes  of  assessing  agency-wide  transition  rates  and  outcomes 
since  fiscal  year  2010.  However,  these  inconsistencies  did  not 
significantly  affect  our  program  selections;  therefore,  these  data  were 
sufficiently  reliable  for  the  purposes  of  selecting  the  10  case  study 
programs. 

To  identify  factors  that  facilitated  technology  transition  within  the  10 
selected  programs,  we  analyzed  DARPA  provided  documentation — 
including  program  briefings,  memorandums  of  agreement,  broad  area 
announcements,  budget  documents,  and  program  completion  reports — 
for  selected  programs  to  identify  factors  that  facilitated  or  precluded  their 
individual  transitions.  We  then  conducted  a  content  analysis  of  these 
individual  factors  to  identify  common  themes  among  the  programs,  which 
led  to  us  determining  that  four  significant  factors  underpinned  technology 
transition  outcomes  in  the  programs  we  reviewed.  Once  we  identified 
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these  four  factors,  we  developed  a  rating  system  to  assess  the  extent  to 
which  each  factor  was  present  in  each  of  our  10  programs,  as  supported 
through  our  analysis  of  program  documentation.  Our  measures  for  each 
of  the  four  factors  were  as  follows: 

Military  or  commercial  demand  for  the  planned  technology 

•  Fully  present:  Demand  for  the  technology  from  a  potential  transition 
partner  existed  throughout  the  program,  which  would  include  (1) 
agreement  between  DARPA  and  a  military  service,  DOD  laboratory  or 
other  warfighter  representatives  that  a  related  military  capability  gap 
or  requirement  exists;  or  (2)  a  private  company  identified  a 
commercial  demand  for  the  technology  or  showed  an  interest  in 
commercializing  it. 

•  Partially  present:  Potential  transition  partners  indicated  to  DARPA  that 
they  believed  a  demand  existed  for  the  technology,  as  is  described 
above,  although  their  interest  was  not  consistent  through  the  end  of 
the  program. 

•  Not  present:  The  factor  did  not  exist  at  all,  and  DARPA  appears  to 
have  initiated  the  program  without  a  potential  transition  partner 
agreeing  that  a  capability  gap  or  potential  commercial  use  existed  at 
any  point  during  the  program. 

Linkage  to  a  research  area  where  DARPA  has  sustained  interest 

•  Fully  present:  In  the  years  preceding  the  program’s  initiation,  at  least 
two  related  DARPA  or  other  DOD  science  and  technology  program 
had  been  completed. 

•  Partially  present:  In  the  years  preceding  the  program’s  initiation,  at 
least  one  related  DARPA  or  other  science  and  technology  program 
had  been  completed  (this  was  the  second  DOD  science  and 
technology  program  of  its  kind). 

•  Not  present:  The  factor  did  not  exist  at  all,  and  this  program  appears 
to  not  have  any  roots  in  previous  similar  DARPA  or  other  DOD 
science  and  technology  programs. 

Active  collaboration  with  potential  transition  partners 

•  Fully  present:  Potential  transition  partners  consistently  participated  in, 
advised,  or  otherwise  supported  the  program. 
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•  Partially  present:  Potential  transition  partners  participated  in,  advised, 
or  otherwise  supported  the  program,  although  their  involvement  was 
not  consistent  through  the  end  of  the  program  or  was  not  present  until 
after  the  prototype  demonstration  (relatively  late  in  the  program). 

•  Not  present:  The  factor  did  not  exist  at  all.  The  program  appears  to 
have  lacked  assistance  from  any  potential  transition  partners,  or  their 
assistance  was  very  infrequent  or  insignificant. 

Achievement  of  clearly  defined  technical  goals 

•  Fully  present:  Measurable  technical  goals  were  set  in  the  program 
and  fully  achieved  to  the  satisfaction  of  DARPA  and  any  transition 
partner  involved  in  the  program,  to  the  extent  that  one  or  more  had 
been  identified  for  the  technology. 

•  Partially  present:  Measurable  technical  goals  were  set  in  the  program, 
but  met  with  varying  levels  of  success.  The  technical  successes 
achieved,  however,  were  sufficient  to  produce  a  technology 
responsive  to  the  interests  of  a  transition  partner,  to  the  extent  that 
one  or  more  had  been  identified  for  the  technology. 

•  Not  present:  Measurable  technical  goals  were  either  not  set  or 
sufficiently  met  in  the  program.  The  level  of  technical  success  was  not 
sufficient  to  produce  a  technology  responsive  to  the  interests  of  a 
transition  partner,  to  the  extent  that  one  or  more  had  been  identified 
for  the  technology. 

Using  this  rating  system,  two  GAO  analysts  analyzed  and  coded  whether 
each  of  the  four  factors  was  fully  present,  partially  present,  or  not  present 
in  each  of  the  10  programs  we  reviewed.  Each  GAO  analyst  coded  all  the 
constituent  items  independently,  and  the  two  analysts  then  met  to  discuss 
and  reconcile  the  differences  between  their  codings.  Following  this  initial 
round  of  coding,  another  GAO  analyst  independently  verified  the 
accuracy  of  the  coding  by  reviewing  the  supporting  program 
documentation.  The  final  assessment  reflected  the  analysts’  consensus 
based  on  the  individual  assessments. 

To  assess  DARPA’s  implementation  of  DOD  policies  and  programs 
intended  to  facilitate  the  transition  of  technologies,  we  identified  and 
analyzed  information  sources  including  policy  instructions,  guidance, 
training  materials,  and  technical  data  repositories  intended  to  promote 
technology  transition  within  DARPA,  DOD  and  the  federal  government. 
We  also  reviewed  previous  federal  directives  issued  by  the  Executive 
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Office  of  the  President  that  were  related  to  technology  transition  at 
DARPA.  We  reviewed  previous  DARPA-sponsored  reports  on  technology 
transition  produced  in  previous  years.  We  reviewed  our  prior  related 
reports  and  program  information  regarding  DOD’s  technology  transition 
programs  and  relevant  DOD  funding  information.  We  reviewed  available 
training,  resources,  and  tools  used  by  DARPA  officials  to  help  bring  about 
technology  transition.  We  reviewed  the  contents  of  DOD  computer 
systems  used  to  disseminate  information  on  DARPA  programs  to 
potential  transition  partners.  We  reviewed  our  prior  reports  and  DOD 
documentation  on  DOD  transition  programs,  including  the  Small  Business 
Innovation  Research,  Small  Business  Technology  Transfer,  and  Joint 
Capability  Technology  Demonstration  programs,  among  others,  to 
understand  the  extent  to  which  DARPA  participates  in  these  programs. 
We  reviewed  the  extent  to  which  DARPA  uses  DOD  transition  funds,  and 
requested  data  regarding  DARPA’s  use  of  small  business  funds  and  its 
technology  transition  outcomes,  although  these  data  were  unavailable  for 
our  analysis,  as  is  discussed  elsewhere  in  this  report.  We  also  reviewed 
historical  information  on  DARPA’s  use  of  DOD  transition  funds  available 
from  public  sources,  including  DOD  budget  documentation. 

To  gather  additional  information  in  support  of  our  review  for  both 
objectives,  we  conducted  interviews  with  current  and  former  officials 
responsible  for  executing,  managing,  and  overseeing  transition  of 
DARPA-developed  technologies,  including  representatives  of  DARPA’s 
senior  leadership  and  Adaptive  Execution  Office,  program  management 
offices  and  selected  program  managers,  military  services  liaisons,  and 
small  business  program  officials.  We  also  interviewed  officials  from  the 
Office  of  the  Assistant  Secretary  of  Defense  for  Research  and 
Engineering  and  DOD’s  Office  of  Small  Business  Programs.  Further,  we 
interviewed  officials  from  selected  military  requirements  and  acquisition 
offices,  including  the  Joint  Staff’s  Force  Structure,  Resource,  and 
Assessment  directorate;  Office  of  the  Deputy  Chief  of  Staff  of  the  Army 
for  Operation,  Plans  and  Training;  Office  of  the  Deputy  Chief  of  Staff  of 
the  Army  for  Logistics;  Office  of  the  Assistant  Secretary  of  the  Army  for 
Acquisition,  Logistics,  and  Technology;  Army  Program  Executive  Offices 
for  Intelligence  Electronic  Warfare  and  Sensors  and  Command,  Control, 
Communications — Tactical;  and  Marine  Corps  Systems  Command.  We 
also  met  with  staff  from  selected  DOD  research  centers,  including  the  Air 
Force  Research  Laboratory  and  the  Office  of  Naval  Research,  and  with 
the  Director  of  Science  and  Technology  curriculum  at  the  Defense 
Acquisition  University. 
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We  conducted  this  performance  audit  from  January  2015  to  November 
2015  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 
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OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 

3030  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3030 


RESEARCH  NOV  0  2  2015 

ANO  ENGINEERING 

Mr.  Michael  J.  Sullivan 

Director.  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street.  N.W. 

Washington.  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability  Office 
(GAO)  Draft  Report,  GAO- 16-5.  "DEFENSE  ADVANCED  RESEARCH  PROJECTS  AGENCY: 
Key  Factors  Drive  Transition  of  Technologies,  but  Better  Training  and  Data  Dissemination  Can 
Increase  Success."  dated  October  2.  2015  (GAO  Code  121260). 

DoD  generally  concurs  with  the  recommendations  of  the  GAO  Draft  report.  However.  DoD 
is  concerned  with  several  findings  that  inaccurately  characterize  DARPA  processes.  In  particular, 
the  findings  that  DARPA  leadership  place  minimal  focus  on  transition  and  that  it  has  delegated 
transition  responsibilities  run  counter  to  recent  DARPA  efforts  to  update  processes  and  procedures 
to  continuously  improve  and  support  these  aims.  DoD  believes  technology  transition  is  a  key  focus 
for  DARPA  during  all  phases  of  a  DARPA  program,  and  assessing  transition  potential  is  an  integral 
part  of  their  review  processes  at  both  the  Technical  Office  and  DARPA  Director  level.  In  2013.  the 
DARPA  Director  increased  the  transition  support  to  Program  Managers  by  directing  the  Adaptive 
Execution  Office  to  serve  as  a  transition  and  engagement  support  element. 

Detailed  comments  on  the  report  recommendations  are  enclosed.  DoD  appreciates  the 
GAO’s  expressed  willingness  to  reassess  these  findings  and  revise  their  characterizations. 


Acting,  Principal  Deputy  ASD(R&E) 
Performing  the  Duties  of  the 
Acting  Secretary  of  Defense  for 
Research  and  Engineering 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  DATED  OCTOBER  2 , 2015 
GAO-16-5  (GAO  CODE  121260) 

“DEFENSE  ADVANCED  RESEARCH  PROJECTS  AGENCY:  KEY  FACTORS  DRIVE 
TRANSITION  OF  TECHNOLOGIES,  BUT  BETTER  TRAINING  AND  DATA 
DISSEMINATION  CAN  INCREASE  SUCCESS” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  I :  To  improve  technology  transition  planning  and  outcomes  at 
DARPA.  GAO  recommends  the  Secretary  of  Defense  direct  the  Director.  DARPA.  to  oversee 
assessments  of  technology  transition  strategies  for  new  and  existing  DARPA  programs  as  part  of 
existing  milestone  reviews  used  to  assess  scientific  and  technical  progress  to  inform  transition 
planning  and  program  changes,  as  necessary'.  GAO  analysis  identified  four  factors  that  could 
underpin  these  assessments,  but  the  uniqueness  of  individual  DARPA  programs  suggests  that 
other  considerations  may  also  be  warranted. 

DoD  RESPONSE:  Partially  concur.  DoD  agrees  technology  transition  assessments  that 
consider  the  four  factors  identified  by  GAO  would  help  inform  DARPA  leadership  program 
decisions.  DoD  disagrees  with  directing  the  Director.  DARPA  to  mandate  assessments  of 
technology  transition  at  DARPA  program  milestone  reviews.  DARPA  leadership  already 
actively  participate  on  technology  transition  discussions  throughout  the  program's  life  to  include 
program  milestone  reviews. 

The  Director.  DARPA  recognizes  the  importance  of  technology  transition,  as  evidenced  by  the 
recent  directive  to  enhance  the  technology  transition  focus  of  the  Adaptive  Execution  Office 
(AF.O).  The  AEO  assists  program  managers  with  engagement  and  technology  transition,  and 
documents  the  ultimate  transition  status  of  programs.  The  transition  assistance  provided  by  the 
AEO  is  further  complemented  by  DARPA’s  staff  liaisons  for  the  Military  Serv  ices. 

RECOMMENDATION  2:  To  improve  technology  transition  planning  and  outcomes  at 
DARPA.  GAO  recommends  the  Secretary  of  Defense  direct  the  Director.  DARPA.  to  increase 
technology  transition  training  requirements  and  offerings  for  DARPA  program  managers, 
leveraging  existing  DoD  science  and  technology  training  curricula,  as  appropriate. 

DoD  RESPONSE:  Partially  concur.  DoD  agrees  that  technology  transition  training  improves 
transition  planning  and  outcomes,  and  that  the  DoD  science  and  technology  training  curricula  is 
a  rich  repository  of  transition  insight.  DoD  disagrees  with  directing  the  Director.  DARPA  to 
increase  training  requirements.  DARPA  program  managers'  relatively  short  tenure  leaves  few 
opportunities  to  expose  them  to  additional  generic  training  offerings.  Considering  the  diverse  and 
complex  nature  of  DARPA  programs.  DoD  believes  DARPA's  current  approach  of  tailored 
curricula  focused  on  a  program's  unique  transition  needs  is  most  appropriate.  DARPA  continues 
to  explore  opportunities  to  draw  from  existing  DoD  and  other  training  materials  to  offer  tailored, 
concise,  and  streamlined  training  to  its  program  managers. 
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RECOMMENDATION  3:  To  improve  technology  transition  planning  and  outcomes  at 
DARPA.  GAO  recommends  the  Secretary  of  Defense  direct  the  Director.  DARPA,  to  increase 
the  dissemination  of  technical  data  on  completed  DARPA  programs  through  Open  Catalog  and 
other  government-sponsored  information  repositories  aimed  at  facilitating  commercialization  of 
technologies. 

DoD  RESPONSE:  Nonconcur.  DoD  disagrees  with  directing  the  Director.  DARPA  to  mandate 
the  use  of  "Open  Catalog"  and  other  government-sponsored  information  repositories.  The  DoD 
intends  to  make  Defense  Technical  Information  Center  (DT1C)  the  central  data  storage  for  all 
DoD  technical  activities  to  include  DARPA  technologies.  DTIC's  technology  transition  mission 
relics  upon  the  network  effect  to  create  the  thick  technology  market  needed  to  effectively  pair 
science  and  technology  offerings  with  transition  partners.  Forcing  both  technology  providers  and 
potential  transition  partners  to  use  multiple  information  repositories  thins  the  DoD  technology 
market  by  spreading  it  across  several  venues,  reducing  the  likelihood  of  finding  a  good  match. 

The  DARPA  Open  Catalog  advocated  in  the  GAO  recommendation  is  still  experimental  and  is 
currently  not  a  reliable  means  for  technology  dissemination.  In  addition,  any  information  entered 
into  this  catalog  must  be  screened  carefully.  DARPA  will  monitor  the  Open  Catalog's  operations 
to  determine  if  and  when  it  should  be  expanded  to  reach  broader  audiences. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates. 

Listen  to  our  Podcasts  and  read  The  Watchblog. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 

71 25,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

w 

Please  Print  on  Recycled  Paper. 

